Forward Selection Initialization Method for Constructive Neural Networks

In this paper we present an initialization method of hidden unit weights for cascade-correlation type constructive neural networks. The forward selection initialization (FSI) method uses a large pool of randomly initialized hidden units and then selects the best one of them by some predetermined criterion which in our simulations was the objective function used in the hidden unit training. The best unit is then trained to the final solution by a desired training algorithm as normally and it is installed in the active network. In total we studied five different algorithms with FSI method that were compared both in classification and regression problems to the basic versions of the same algorithms. The investigated algorithms were Cascade-Correlation, Modified Cascade-Correlation, Cascade, Cascade Network, and Fixed Cascade Error. The simulation results show that the proposed initialization method is beneficial not only in rather simple but also in highly complicated problems when compared to the corresponding algorithms using only single randomly initialized candidate units in the hidden unit